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Mission 

Our  mission  is  to  provide  independent,  relevant,  and  timely  oversight 
of  the  Department  of  Defense  that  supports  the  warfighter;  promotes 
accountability,  integrity,  and  efficiency;  advises  the  Secretary  of 
Defense  and  Congress;  and  informs  the  public. 


Vision 

Our  vision  is  to  be  a  model  oversight  organization  in  the  Federal 
Government  by  leading  change,  speaking  truth,  and  promoting 
excellence — a  diverse  organization,  working  together  as  one 
professional  team,  recognized  as  leaders  in  our  field. 
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Objective 

The  audit  objective  was  to  determine 
whether  Defense  Logistics  Agency  (DLA] 
personnel  are  adequately  processing 
product  quality  deficiency  reports  and 
identifying  the  root  cause  for  defective 
spare  parts.  This  is  the  first  in  a  series  of 
audits  on  DLA  processing  product  quality 
deficiency  reports. 

Finding 

DLA  Aviation  quality  assurance  personnel 
conducted  adequate  investigations  for 
product  quality  deficiency  reports. 

However,  they  did  not  adequately  process 
21  of  52  that  we  non-statistically  sampled 
and  properly  code  them  to  reflect  the  root 
causes  of  the  deficiencies  determined  by 
their  investigations.  This  occurred  because: 


August  2013  and  August  2014.  The  coding  differed  because 
of  deficiencies  in  the  processes  Military  Department  screening 
points  used  to  update  information  in  the  systems  and 
outdated  software  code. 

The  inaccurate  data  limits  the  effectiveness  of  the  DoD 
product  quality  deficiency  report  program  and  prevents 
meaningful  analysis  of  the  primary  causes  of  spare-part 
quality  deficiencies.  In  addition,  the  inaccurate  data  weakens 
DoD's  ability  to  hold  contractors  responsible  for  providing 
defective  parts  because  contractor  evaluation  systems  contain 
incomplete  data.  Ultimately,  this  increases  the  risk  of  DoD 
procuring  nonconforming  spare  parts  from  contractors,  which 
impacts  warfighter  readiness  and  safety. 

Recommendations 

We  recommend  that  the  Director,  DLA,  develop  an  action  plan 
with  milestones  to  improve  product  quality  deficiency  report 
processing.  The  plan  should  address  the  problems  that  this 
report  identified  and: 


•  quality  assurance  personnel 
lacked  sufficient  guidance  to  make 
appropriate  coding  decisions  and  did 
not  have  a  complete  understanding 
of  how  their  coding  actions  impacted 
contractor's  quality  ratings; 

•  supervisors  failed  to  conduct  adequate 
reviews  of  product  quality  deficiency 
report  investigations;  and 

•  the  product  quality  deficiency  report 
program  lacked  adequate  oversight  to 
improve  operational  effectiveness. 

In  addition,  the  cause  codes  assigned  in 
deficiency  reporting  systems  differed  for 
17  of  the  52  sampled  investigations  and 
for  a  total  of  1,921  of  the  9,347  reports 
that  the  DLA  Supply  Chains  closed  between 
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•  update  existing  guidance  on  product  quality  deficiency 
report  processing,  coding  decisions,  and  the  associated 
supervisory  reviews; 

•  develop  procedures,  controls,  and  associated  metrics 
that  evaluate  deficiency  reporting  results  to  improve 
operational  effectiveness;  and 

•  require  coordination  with  deficiency  reporting  system 
program  offices  on  the  sufficiency  of  planned  corrective 
actions  and  establish  procedures  to  ensure  that  codes 
are  consistent  between  deficiency  reporting  systems. 

Management  Comments  and 
Our  Response 

Comments  from  the  Director,  DLA  Logistics  Operations,  did 
not  address  all  specifics  of  one  of  the  four  recommendations, 
and  further  comments  are  required  on  Recommendation  l.c(2] 
by  August  3,  2015.  Please  see  the  Recommendations  Table  on 
the  back  of  this  page. 
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Recommendations  Table 


Management 

Recommendations 
Requiring  Comment 

No  Additional 
Comments  Required 

Director,  Defense  Logistics  Agency 

l.c(2) 

l.a,  l.b,  l.c(l) 

Please  provide  Management  Comments  by  August  3,  2015. 
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MEMORANDUM  FOR  UNDER  SECRETARY  OF  DEFENSE  FOR  ACQUISTION, 

TECHNOLGY,  AND  LOGISTICS 
DIRECTOR,  DEFENSE  LOGISTICS  AGENCY 

SUBJECT:  Defense  Logistics  Agency  Can  Improve  Its  Product  Quality  Deficiency  Report 
Processing  (Report  No.  DODIG-2015-140) 

We  are  providing  this  draft  report  for  review  and  comment.  This  is  the  first  in  a  series 
of  audits  on  Defense  Logistics  Agency  processing  product  quality  deficiency  reports. 

Defense  Logistics  Agency  quality  assurance  personnel  did  not  adequately  process  product 
quality  deficiency  reports  and  properly  code  them  to  reflect  the  root  causes  of  the  deficiencies 
determined  by  their  investigations.  In  addition,  coding  discrepancies  existed  between 
deficiency  reporting  systems.  We  conducted  this  audit  in  accordance  with  generally  accepted 
government  auditing  standards. 

We  considered  management  comments  on  a  draft  of  this  report  when  preparing  the  final 
report.  DoD  Instruction  7650.03  requires  that  recommendations  be  resolved  promptly. 
Comments  from  the  Director,  Defense  Logistics  Agency  Logistics  Operations,  responding  for 
the  Director,  Defense  Logistics  Agency,  for  Recommendations  l.a,  l.b,  and  l.c(l)  conformed 
to  the  requirements  of  DoD  Instruction  7650.03;  therefore,  we  do  not  require  additional 
comments.  The  Director  agreed  with  Recommendation  l.c(2)  but  did  not  describe  corrective 
actions  to  address  the  recommendation.  Therefore,  we  request  additional  comments  on 
Recommendation  l.c(2)  by  August  3,  2015  that  include  the  actions  the  Defense  Logistics 
Agency  will  take. 

Please  send  a  PDF  file  containing  your  comments  to  audcolu@dodig.mil.  Copies  of  your 
comments  must  have  the  actual  signature  of  the  authorizing  official  for  your  organization. 

We  cannot  accept  the  /Signed/  symbol  in  place  of  the  actual  signature.  If  you  arrange  to  send 
classified  comments  electronically,  you  must  send  them  over  the  SECRET  Internet  Protocol 
Router  Network  (SIPRNET). 

We  appreciate  the  courtesies  extended  to  the  staff.  Please  direct  questions  to  me  at 
(703)  604-9077  (DSN  664-9007). 


leline  L.  Wicecarver 
Assistant  Inspector  General 
Acquisition,  Parts,  and  Inventory 
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Introduction 


Objective 

The  audit  objective  was  to  determine  whether  Defense  Logistics  Agency  (DLA) 
personnel  are  adequately  processing  product  quality  deficiency  reports  (PQDRs) 
and  identifying  the  root  cause  of  deficiencies  in  spare-part  quality.  This  is  the 
first  in  a  series  of  audits  on  DLA  PQDR  processing.  This  audit  focused  on  PQDRs 
processed  by  the  DLA  Aviation  Supply  Chain.  The  next  audit  in  this  series  will 
determine  whether  DLA  Aviation  is  holding  contractors  responsible  for  producing 
deficient  parts  and  obtaining  adequate  restitution.  See  Appendix  A  for  additional 
details  on  our  scope  and  methodology,  use  of  computer  processed  data,  and  prior 
coverage  of  PQDRs. 

Background 

Defense  Logistics  Agency 

DLA,  headquartered  at  Fort  Belvoir,  Virginia,  provides  the  Army,  Marine  Corps, 
Navy,  and  Air  Force,  and  combined  allied  forces  with  a  full  spectrum  of  logistics, 
acquisition,  and  technical  services,  including  supplying  more  than  85  percent1  of 
the  military’s  spare  parts. 

DLA  Aviation,  headquartered  in  Richmond,  Virginia,  is  the  U.S.  military’s  integrated 
materiel  manager  for  more  than  1.1  million  repair  parts  and  operating  supply  items 
in  support  of  all  fixed-and  rotor-wing  aircraft,  including: 

•  spares  for  engines  on  fighters,  bombers,  transports  and  helicopters; 

•  all  airframe  and  landing  gear  parts; 

•  flight  safety  equipment;  and 

•  propeller  systems. 

In  addition  to  DLA  Aviation,  DLA  has  several  other  Supply  Chains  that 
process  PQDRs. 

Product  Quality  Deficiency  Reporting  Process 

PQDRs  are  the  primary  tool  for  feedback  on  the  quality  of  items  issued  through 
the  supply  chain  or  field  level  activity.  They  are  submitted  when  new  or  newly 
reworked  Government-owned  products  are  determined  not  to  fulfill  their  expected 
purpose,  operation,  or  service  due  to  any  or  all  of  the  following: 


1  Source:  www.dla.mil. 
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•  deficiencies  in  design; 

•  specification; 

•  materiel; 

•  software; 

•  manufacturing  process;  or 

•  workmanship. 

Personnel  generate  a  PQDR  as  either  a  category  I  or  category  II,  based  on  the 
nature  of  the  deficiency. 

•  Category  I — a  product  quality  deficiency  that  may: 

o  cause  death,  injury,  severe  occupational  illness,  or  major  loss  or 
damage  to  a  weapon  system; 

o  critically  restrict  the  combat-readiness  capabilities  of  the  using 
organization  or  results  in  a  production  line  stoppage. 

•  Category  II — a  product  quality  deficiency  that  does  not  meet  the  criteria 
set  forth  in  category  I. 

DLA  Regulation  41  55. 242  implements  DoD  policy  for  reporting  of  product  quality 
deficiency  data.  The  Regulation  establishes  a  system  for  feedback  on  product 
quality  and  provides  guidance  for  the  initial  reporting,  cause  correction,  and  status 
accounting  of  individual  product  quality  deficiencies.  It  also  specifies  that  DoD 
organizations  should  use  the  data  gathered  from  the  PQDR  program  to  identify 
problems,  trends,  and  recurring  deficiencies  in  spare-part  quality.  The  process 
primarily  focuses  on  the  following  four  roles. 

•  Originator — a  user  who  discovers  the  defective  product  and  initiates 
the  PQDR  and,  in  some  cases,  provides  the  deficient  part  (an  exhibit)  for 
Government  or  contractor  testing. 

•  Screening  Point — a  designated  activity  identified  within  each 
DoD  organization  that  reviews  the  PQDR  for  validity,  accuracy,  and 
completeness  of  required  information  and  identifies  and  transmits  the 
PQDR  to  the  proper  action  point  within  or  outside  the  DoD  organization. 

•  Action  Point — leads  and  manages  the  PQDR  investigation  and,  for 
DLA-managed  items,  this  responsibility  is  assigned  to  a  quality 
assurance  specialist. 

•  Support  Point — assists  the  action  point  in  the  investigation  upon  request. 
This  is  generally  the  Defense  Contract  Management  Agency  (DCMA). 


2  DLA  Regulation  4155.24,  "Product  Quality  Deficiency  Report  Program,"  July  20, 1993. 
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Figure  1  identifies  the  DoD  organizations  that  fulfilled  those  roles  for  the  PQDRs 
reviewed  during  this  audit.3 

Figure  1.  Organizations  Involved  in  Processing  DLA  Aviation  PQDRs 


•  DoD 

Maintenance 

Organization 


•  DoD 
Inventory 
Control  Point 


•  DLA  Aviation 


DoD  Computer  Systems  Used  to  Support  PQDR  Processing 

DoD  organizations  document  PQDR  processing  and  resolution  results  in  the 
U.S.  Navy  hosted  Product  Data  Reporting  and  Evaluation  Program  (PDREP) 
information  system.  DLA  personnel  process  PQDRs  in  the  DLA  Enterprise  Business 
System  (EBS)  and  other  DoD  organizations  use  the  Joint  Deficiency  Reporting 
System  (JDRS)  to  submit  PQDRs  for  aviation  related  parts.  As  shown  in  Figure  2, 
PDREP  shares  and  receives  information  with  these  other  DoD  systems  during 
the  PQDR  process.  At  the  conclusion  of  the  investigative  process,  PQDR  data  is 
transmitted  to  the  Past  Performance  Information  Retrieval  System  (PPIRS). 


Figure  2.  Flow  of  Spare  Part  Quality  Data  Through  Deficiency  Reporting  Systems 


During  the  PQDR  investigation,  quality  assurance  personnel  assign  codes  and 
enter  text  into  the  various  systems  to  identify  the  cause  of  the  deficiency,  the 
party  responsible  for  the  deficiency  (contractor  or  government),  actions  taken  to 
correct  the  deficiency,  and  the  disposition  of  the  defective  product.  Table  1  lists 
the  codes  available  in  EBS,  JDRS  and  PDREP  for  categorizing  the  causes  of  spare 
part  deficiencies. 


3  See  Appendix  B  for  a  detailed  description  of  the  PQDR  process  when  DLA  is  the  Action  Point. 
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Table  1.  Cause  Codes  Available  in  DoD  Deficiency  Reporting  Systems 


Code 

Definition 

System  Availability  (EBS,  JDRS,  PDREP) 

C 

Contract  Error 

All 

D 

Technical  Data  Package/Design  Error 

All 

M 

Maintenance  Error 

JDRS  &  PDREP  Only 

N 

Contractor  Noncompliance 

All 

P 

Part  Application 

JDRS  &  PDREP  Only 

S 

Shelf  Life  Item 

JDRS  &  PDREP  Only 

U 

Misuse  of  Item 

All 

X 

Undetermined 

All 

Z 

Not  Applicable 

JDRS  &  PDREP  Only 

DLA  and  other  Federal  Government  agencies  use  PPIRS  to  track  contractor 
past  performance  information  (timeliness  of  contractor  delivery  and  spare  part 
quality),  which  is  used  to  make  future  contract  award  decisions.  PQDRs  impact 
a  contractor's  PPIRS  quality  rating  when  the  contractor  is  identified  as  having 
provided  deficient  parts.4 

Review  of  Internal  Controls 

DoD  Instruction  5010.40,  "Managers’  Internal  Control  Program  Procedures," 

May  30,  2013,  requires  DoD  organizations  to  implement  a  comprehensive 
system  of  internal  controls  that  provides  reasonable  assurance  that  programs 
are  operating  as  intended  and  to  evaluate  the  effectiveness  of  the  controls.  We 
identified  internal  control  weaknesses  where  DLA  Aviation  quality  assurance 
personnel  did  not  adequately  process  21  of  the  52  sampled  PQDR  investigations 
and  properly  code  them  to  reflect  the  root  causes  of  the  deficiencies  determined 
by  their  investigations.  In  addition,  the  cause  codes  assigned  in  EBS  and  PDREP 
differed  for  17  of  the  52  sampled  PQDR  investigations  and  for  a  total  of  1,921  of 
the  9,347  PQDRs  DLA  closed  during  a  one  year  period. 

We  will  provide  a  copy  of  the  report  to  the  DLA  senior  official  responsible  for 
internal  controls. 


4  See  Appendix  C  for  details  on  the  PPIRS  quality  rating  process. 
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Finding 

Defense  Logistics  Agency  Aviation  Did  Not  Adequately 
Process  Product  Quality  Deficiency  Reports 

DLA  Aviation  quality  assurance  personnel  conducted  adequate  investigations  for 
49  of  the  52  we  non-statistically  sampled.  However,  personnel  did  not  select  the 
right  code  to  properly  identify  the  root  causes  of  the  deficiencies  as  determined  by 
their  investigations  for  21  of  the  52  PQDR  investigations. 

This  occurred  because: 

•  quality  assurance  personnel  lacked  sufficient  guidance  to  make 
appropriate  coding  decisions  and  did  not  have  a  complete  understanding 
of  how  their  coding  actions  impacted  contractors’  quality  ratings, 

•  supervisors  did  not  sufficiently  review  quality  assurance  specialists’  PQDR 
investigation  results  and  associated  coding  actions,  and 

•  DLA  failed  to  establish  a  formal  system  to  adequately  monitor  and 
improve  the  operational  effectiveness  of  the  PQDR  program. 

In  addition,  the  cause  codes  assigned  in  EBS  and  PDREP  differed  for  17  of  the 
52  sampled  PQDR  investigations  and  for  a  total  of  1,921  of  the  9,347  PQDRs  that 
DLA  Supply  Chains  closed  between  August  2013  and  August  2014.  The  systems 
contained  different  cause  codes  because  of  deficiencies  in  processes  the  Military 
Department  screening  points  used  to  update  information  in  the  systems  and  the 
U.S.  Navy’s  failure  to  remove  outdated  software  code. 

The  inaccurate  data  limits  the  effectiveness  of  the  DoD  PQDR  Program  and 
prevents  meaningful  analysis  of  the  primary  causes  of  spare  part  quality 
deficiencies.  In  addition,  the  inaccurate  data  weakens  DLA’s  ability  to  hold 
contractors  responsible  for  providing  non-conforming  parts  because  contractor 
evaluation  tools  such  as  PPIRS  contain  incomplete  data.  Ultimately,  this  increases 
the  risk  of  DoD  procuring  non-conforming  spare  parts  from  contractors,  which 
impacts  warfighter  readiness  and  safety. 
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Defense  Logistics  Agency  Aviation  Product  Quality 
Deficiency  Report  Processing 

DLA  Aviation  closed  1,102  PQDRs  during  the  period  we  reviewed.5  Table  2  shows 
that  for  658  (60  percent)  of  those  PQDRs,  DLA  quality  assurance  personnel  selected 
a  cause  code  that  indicated  that  they  could  not  determine  what  specifically  caused 
the  defective  parts. 

Table  2.  Audit  Population  of  DLA  Aviation  PQDRs  by  Cause  Code 


Code 

Cause  Code  Description 

PQDR  Count 

Percent  of  Total 

X/Z 

Undetermined/Not  Applicable 

658 

60 

N 

Contractor  Noncompliance 

232 

21 

D 

Technical  Data  Package/Design  Error 

109 

10 

C 

Contract  Error 

47 

4 

M 

Maintenance  Deficiency 

37 

3 

p/s/u 

Part  Application/Shelf  Life/Misuse  of  Item 

19 

2 

Total 

1,102 

100 

We  selected  and  tested  a  non-statistical  sample  of  52  unique  PQDR  investigations 
from  the  audit  population  of  DLA  Aviation  PQDRs.  The  large  number  of  PQDRs 
that  lacked  specific  causes  raised  concerns  about  the  adequacy  of  the  PQDR 
investigations.  Therefore,  as  shown  in  Table  3,  the  majority  of  the  PQDRs  we 
selected  were  from  those  categories. 


Table  3.  Sampled  DLA  Aviation  PQDRs  by  Cause  Code 


Code 

Definition 

Sampled  PQDR  Count 

x/z 

Undetermined/Not  Applicable 

40 

D 

Technical  Data  Package/Design  Error 

7 

C/M/U 

Contract  Error/Maintenance  Error/Misuse  of  Item 

5 

Total 

52 

Based  on  our  testing,  we  determined  that  it  was  not  the  adequacy  of  the 
investigations  that  caused  a  large  number  of  PQDRs  to  be  classified  as 
undetermined  or  not  applicable.  Instead,  we  determined  that  quality  assurance 
personnel  did  not  select  the  right  code  to  properly  identify  the  root  causes  of  the 
deficiencies  as  determined  by  their  investigations. 


5  This  period  of  review  was  August  2013  through  February  2014  (See  Appendix  A  for  details). 
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Quality  Assurance  Personnel  Performed 
Adequate  Investigations 

DLA  Aviation  quality  assurance  personnel  conducted  adequate  investigations  for 
49  of  the  52  PQDR  investigations  we  reviewed.  To  determine  this,  we  reviewed  the 
steps  to  investigate  the  deficiencies  and  interviewed  the  responsible  DLA  Aviation 
quality  assurance  personnel  to  ensure  that  they  performed  sufficient  work  in  the 
course  of  their  investigations  to  either  determine  a  root  cause  or  to  justify  that  one 
could  not  be  determined. 

An  adequate  PQDR  investigation  includes  identifying  whether  parts  reported  are 
defective  and  why.  The  investigation  generally  involves  DLA  quality  assurance 
personnel  coordinating  with  the  originator  to  have  a  deficient  part  sent  to  the 
contractor  or  government  lab  for  testing.  DCMA  quality  assurance  personnel  may 
also  assist  in  the  investigation  if  they  are  involved  with  administering  the  contract 
for  the  spare  parts.  Additional  steps  in  the  investigation: 

•  determine  if  the  customer  will  receive  credit  for  the  defective  part; 

•  input  findings  and  recommendations  of  investigation  codes  into  EBS; 

•  prepare  a  closing  report;  and 

•  provide  disposition  instructions  for  the  deficient  parts. 

Quality  Assurance  Personnel  Did  Not  Assign  Accurate 
Cause  Codes  and  Codes  Differed  Between  Deficiency 
Reporting  Systems 

DLA  Aviation  quality  assurance  personnel  did  not  consistently  choose  the 
appropriate  codes  to  identify  the  root  cause  of  the  deficiencies  determined  by 
their  investigations.  Specifically,  DLA  Aviation  quality  assurance  specialists 
assigned  inaccurate  cause  codes  for  21  of  the  52  investigations  we  reviewed. 

We  observed  that  DLA  Aviation  quality  assurance  personnel  were  especially  likely 
to  inaccurately  record  a  code  to  reflect  that  the  cause  was  undetermined  when 
their  investigations  determined  that  a  contractor  was  at  fault  for  the  deficiency. 

For  these  PQDRs,  the  quality  assurance  specialists  should  have  assigned  a  cause 
code  to  reflect  contractor  noncompliance.  This  situation  occurred  in  11  of  the 
21  coding  errors. 

We  also  found  that  the  cause  codes  DLA  Aviation  quality  assurance  personnel 
assigned  in  EBS  were  not  always  accurately  reflected  in  PDRER  Specifically,  the 
cause  codes  assigned  for  17  of  the  52  investigations  we  reviewed  did  not  match 
between  the  two  systems.  We  compared  the  appropriateness  of  the  codes  for 
these  sample  investigations  and  determined  that  generally  the  EBS  codes  more 
appropriately  reflected  the  actual  cause  determined  in  the  investigation  rather 
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than  the  PDREP  codes.  We  further  examined  coding  data  and  determined  that 
cause  codes  assigned  in  EBS  and  PDREP  differed  for  1,921  of  the  9,347  PQDRs 
(21  percent)  that  DLA  Supply  Chains  closed  between  August  2013  and  August  2014. 

Policy  and  Controls  Over  Product  Quality  Deficiency 
Report  Processing  Needs  Improvement 

DLA  policy  did  not  include  sufficient  guidance  to  enable  quality  assurance 
personnel  to  make  appropriate  coding  decisions  or  provide  them  with  sufficient 
information  on  how  their  coding  decisions  impacted  a  contractor's  quality  rating. 

In  addition,  supervisors  did  not  sufficiently  review  quality  assurance  specialists' 
PQDR  investigation  results  and  associated  coding  actions.  DLA  also  failed  to 
establish  a  formal  system  to  adequately  monitor  and  improve  the  operational 
effectiveness  of  the  PQDR  program. 

DLA  Guidance  Did  Not  Adequately  Define  Codes  That 
Identified  the  Cause  of  Spare-Part  Deficiencies 

DLA  guidance  did  not  adequately  define  the  most  appropriate  codes  DLA  Aviation 
quality  assurance  specialists  should  use  to  categorize  the  causes  of  defective  spare 
parts.  Upon  completing  their  investigation,  DLA  quality  assurance  specialists’  must 
assign  one  of  five  available  cause  codes  as  part  of  the  process  to  close  the  PQDR 
in  EBS.6  Although  EBS  maintained  a  drop  down  list  of  available  cause  codes,  it 
did  not  maintain  detailed  definitions  or  provide  business  scenarios  and  examples 
to  clearly  demonstrate  which  codes  DLA  personnel  should  assign  to  accurately 
identify  the  root  cause  of  defective  parts.  Instead,  code  definitions  were  listed 
in  DLA  Regulation  4155.24.  The  Regulation  briefly  defines  the  five  broad  cause 
codes  identified  in  EBS  as  well  as  four  additional  cause  codes  available  in  PDREP. 
However,  the  Regulation  does  not  provide  specific  examples  or  sufficiently  explain 
the  circumstances  when  specific  codes  should  be  selected. 

The  deficiencies  with  the  guidance  contributed  to  coding  errors.  For  example, 
we  reviewed  a  Navy  PQDR  for  a  deficient  seal  that  DLA  sold  for  $4,648  each. 

The  DLA  Aviation  quality  assurance  specialist  who  performed  the  investigation 
erroneously  selected  Cause  Code  C  (contract  error)  in  EBS  when  closing  the  quality 
report  because  he  mistakenly  thought  it  reflected  contractor  error,  meaning  the 
deficiency  was  the  contractor’s  fault.  However,  Cause  Code  C  actually  represents 
an  error  that  government  personnel  made  in  the  writing  of  the  contract,  such  as 
including  the  wrong  part  number  or  wrong  specifications  in  the  contract.  In  this 
case,  Cause  Code  N  (contractor  noncompliance),  which  represents  an  error  a 


See  Table  1  on  page  4  of  this  report  for  a  complete  list  of  the  codes  and  their  associated  definitions. 
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contractor  made  when  manufacturing  the  part,  would  have  been  right  code  to 
accurately  summarize  the  conclusions  of  this  investigation.  DLA  Aviation  quality 
assurance  personnel  acknowledged  that  it  is  common  for  personnel  to  mistakenly 
assign  the  contract  error  cause  code  instead  of  the  contractor  noncompliance  cause 
code  in  these  situations. 


In  another  example,  we  reviewed  a  Navy  PQDR  for  a  defective  aircraft  lever  that 
DLA  sold  for  $1,541  each.  The  part  is  a  critical  safety  item  and  renders  the  aircraft 
not  fully  mission  capable  until  suitable  replacement  parts  are  provided.  The 
DLA  Aviation  quality  assurance  specialist  that  conducted  the  investigation  for  this 
PQDR  selected  Cause  Code  D  (technical  data  package/design  error).  Cause  Code  D 
is  appropriate  when  the  contractor  produced  the  part  accurately  to  the  technical 
drawings,  but  the  Government  cited  the  wrong  drawings  in  the  contract  or  the 
drawings  were  in  error.  However,  in  this  case,  the  quality  assurance  specialist’s 
investigation  determined  that  during  the  machining  of  the  lever,  the  contractor’s 
production  machine  malfunctioned  resulting  in  the  deficiency. 


The  quality 
assurance 

specialist  did  a  good 
job  investigating  the 
cause  of  the  deficiency 
but  failed  to  correctly 
identify  and  assign 
the  appropriate 
cause  code. 


The  quality  assurance  specialist  did  a  good  job 

investigating  the  cause  of  the  deficiency  but  failed  to 
correctly  identify  and  assign  the  appropriate  cause 
code  to  reflect  that  contractor  noncompliance  had 
caused  the  defect.  When  we  brought  this  to  the 
quality  assurance  specialist’s  attention,  he  stated 
that  the  manufacturer  had  produced  this  item  for 
several  years  without  any  problems  and  that  he 
did  not  see  the  failure  of  the  contractor’s  machine  as 
contractor  noncompliance. 


DLA  should  update  its  policy  to  allow  for  quality  assurance  personnel  to  easily 
obtain  code  definitions  and  business  scenarios  to  identify  and  assign  appropriate 
codes  when  processing  PQDRs. 


Quality  Assurance  Specialists  Did  Not  Fully  Understand  How 
Their  Coding  Decisions  Impacted  Contractor  Quality  Ratings 

DLA  Aviation  quality  assurance  specialists  also  miscoded  the  causes  for  deficient 
parts  because  they  did  not  have  a  complete  and  accurate  understanding  of  how 
their  coding  decisions  impacted  a  contractor’s  quality  rating.  Several  quality 
assurance  specialists  we  interviewed  stated  they  did  not  assign  Cause  Code  N  to 
identify  contractor  noncompliance  because  they  did  not  want  to  adversely  impact 
the  contractor’s  quality  rating  based  on  the  circumstances. 
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This  occurred  for  a  variety  of  reasons  that  are  discussed  in  the  following 
paragraphs  and  DLA  Regulation  4155.24  lacked  specific  guidance  to  inform  DLA 
quality  assurance  specialists  on  how  to  handle  those  scenarios.  In  addition, 
information  we  obtained  from  a  PPIRS  official  on  how  PPIRS  calculates  contractor’s 
quality  ratings  revealed  that  DLA  Aviation  quality  assurance  specialists  did 
not  have  a  complete  and  accurate  understanding  of  how  their  coding  decisions 
impacted  a  contractor’s  quality  rating. 

For  example,  we  reviewed  a  DLA  Aviation  quality  assurance  specialist’s 
investigation  of  an  Air  Force  PQDR  for  deficient  C-135  aircraft  structural 
components,  shown  in  Figure  3,  which  DLA  sold  for  $2,919  each. 


Figure  3.  C-135  Structural  Components  (Note:  Bottom  part  manufactured  too  long) 
Source:  PDREP 


The  deficient  parts  caused  a  work  stoppage  at  an  Air  Force  maintenance  facility. 
DCMA  personnel  supported  the  investigation  and  concluded  that  the  contractor 
did  not  make  the  parts  according  to  the  applicable  technical  drawing.  However, 
the  quality  assurance  specialist  input  a  code  to  reflect  that  the  cause  was 
undetermined  because  the  contractor  agreed  to  repair/replace  the  parts  and  the 
quality  assurance  specialist  did  not  want  to  adversely  impact  the  contractor’s 
quality  rating. 
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To  understand  how  the  PQDR  coding  impacted  contractor’s  quality  ratings,  we 
contacted  a  PPIRS  official  responsible  for  quality  assurance.  The  PPIRS  official 
informed  us  that  the  cause  code  does  not  directly  impact  a  contractor’s  quality 
rating.  Instead,  PPIRS  uses  another  code  that  the  quality  assurance  specialist 
assigns  during  PQDR  close-out,  the  defect  responsibility  code,  to  calculate  a 
contractor’s  quality  rating  when  the  code  reflects  contractor  responsibility.  The 
defect  responsibility  code  identified  the  organization  responsible  for  causing  the 
defective  parts.7  In  addition,  the  PPIRS  official  informed  us  that  PPIRS  also  used 
the  total  number  of  PQDRs  compared  to  the  total  number  of  delivery  records  to 
calculate  the  contractor’s  quality  rating. 

Table  4  lists  the  defect  responsibility  codes  and  whether  or  not  PPIRS  used  them  to 
calculate  contractor  quality  ratings. 


Table  4.  PQDR  Defect  Responsibility  Codes  and  PPIRS  Usage 


Code 

Definition 

Used  In  PPIRs  Quality  Rating  Calculation 

A 

Contractor 

Yes 

B 

Procurement  Agency 

No 

C 

Organic  Manufacturing 

No 

H 

Undetermined 

No 

1 

Invalid  Report 

No 

U 

Government  Using  Activity 

No 

Although  the  quality  assurance  specialist  assigned  an  undetermined  cause  code 
that  would  not  impact  the  contractor’s  quality  rating,  his  assignment  of  the  defect 
responsibility  code  of  "A"  (private  contractor)  actually  allowed  for  the  PQDR  to 
meet  the  PPIRS  criteria  and  was  used  to  calculate  the  contractor’s  quality  rating. 


7  See  Appendix  C  for  details  on  the  PPIRS  rating  process. 
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In  another  instance,  we  reviewed  a  DLA  Aviation  quality  assurance  specialist’s 
investigation  of  an  Air  Force  PQDR  for  deficient  C-5A  Flap  Assemblies,  shown  in 
Figure  4,  which  DLA  sold  for  $33,083  each. 


In  this  case,  a  contractor 
subcontracted  the  manufacturing 
to  another  contractor.  However, 
the  subcontractor  milled  the 
fastener  edge  distance  to  5/16  of 
an  inch  when  the  contract 
specifications  required  them  to 
be  11/16  of  an  inch.  As  a  result, 
the  fasteners  were  too  short 
and  if  installed,  could  cause  the 
aircraft  to  operate  unsafely. 

Although  Cause  Code  N  was 
the  appropriate  code  to  assign 
for  these  circumstances,  the  quality  assurance  specialist  input  a  code  to  reflect 
that  he  could  not  determine  the  cause  of  the  defect  because  he  was  hesitant  and 
believed  it  inappropriate  to  blame  the  contractor  for  its  subcontractor’s  defective 
parts.  In  addition,  the  quality  assurance  specialist  assigned  a  defect  responsibility 
code  of  "H"  (unknown).  Therefore,  by  not  appropriately  coding  the  PQDR  and 
assigning  responsibility  to  the  contractor,  the  quality  assurance  specialist  did 
not  allow  for  the  PQDR  to  meet  the  PPIRS  criteria  for  calculating  the  contractor’s 
quality  rating. 

In  another  example,  we  reviewed 
a  DLA  Aviation  quality  assurance 
specialist’s  investigation  of  an 
Air  Force  PQDR  for  deficient 
F-4  aircraft  parachute  canopy 
assemblies,  shown  in  Figure  5, 
which  DLA  sold  for  $896  each. 

In  this  case,  the  quality  assurance 
specialist  chose  not  to  assign  a 
cause  code  to  reflect  contractor 
noncompliance  even  though  his 
investigation  revealed  that  the 
contractor  produced  deficient 
material.  The  quality  assurance 
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specialist  stated  that  the  contractor’s  quality  rating  had  already  been  negatively 
impacted  because  quality  deficiencies  were  already  identified  with  other  canopy 
assemblies  obtained  under  the  same  contract.  Therefore,  he  did  not  think  it  was 
appropriate  to  impact  the  score  with  additional  deficiencies. 

Overall,  it  is  important  that  DLA  quality  assurance  specialists  appropriately 
code  PQDRs  to  identify  contractor  noncompliance  when  they  either  produce 
defective  parts  or  fail  to  ensure  the  quality  of  parts  received  from  subcontractors. 
Appropriate  codes  provide  an  incentive  to  contractors  to  ensure  they  provide 
high-quality  parts  and  alert  DoD  of  contractors  who  cannot  meet  the  standard. 

DLA  should  update  its  PQDR  guidance  to  ensure  that  quality  assurance  specialists 
assign  codes  to  reflect  contractor  noncompliance  and  responsibility  when 
warranted  and  specify  in  PQDR  guidance  how  the  quality  assurance  specialists’ 
coding  decisions  impact  contractors’  quality  ratings  in  PPIRS. 


Quality  Assurance  Personnel  Did  Not  Conduct  Adequate 
Supervisory  Reviews 


DLA  Aviation  quality  assurance  supervisors  failed  to 
conduct  adequate  supervisory  reviews  of  the  PQDR 
investigation  results  and  associated  coding  actions. 
DLA  policy  requires  a  supervisory  review  of  the 
PQDR  closing  letter,  which  contains  the  cause 
for  the  reported  deficiency.  The  responsible 
quality  assurance  specialist  must  assign  a  cause 
code  before  closing  a  PQDR.  If  the  root  cause  is 
undetermined  then  the  quality  assurance  specialist 
must  justify  why  the  specific  cause  could  not  be 
determined.  The  quality  assurance  specialist  should 


DLA 

Aviation 

W  quality  assurance  ^ 
J  supervisors  failed  ’ 


to  conduct  adequate 
supervisory  reviews  of 
the  PQDR  investigation 
results  and  associated 
coding  actions. 


complete  several  steps  to  determine  the  root  cause  of  the  deficiency.  These  steps 


include  a  review  of  the  contractual  information,  technical  and  quality  history 


files,  and  DCMA  report.  An  adequate  supervisory  review  should  ensure  that  the 


responsible  quality  assurance  specialist  performs  these  steps  before  completing  the 


PQDR  investigation  and  closing  letter. 


DLA  Aviation  personnel  did  not  consistently  conduct  adequate  supervisory 
reviews  of  the  quality  assurance  specialist’s  investigations.  We  found  that 
quality  assurance  specialists  did  not  adequately  investigate  3  of  the  52  PQDR 
investigations  we  reviewed.  In  addition,  our  review  found  that  DLA  Aviation 
quality  assurance  personnel  did  not  choose  the  appropriate  codes  to  identify  the 
root  cause  of  the  deficiencies  determined  by  21  of  the  52  PQDR  investigations.  We 
believe  that  many  of  those  deficiencies  would  have  been  identified  had  supervisors 
performed  adequate  reviews. 
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To  illustrate,  we  reviewed  a  DLA  Aviation  quality  assurance  specialist's 
investigation  of  an  Air  Force  PQDR  for  a  B-52H  aircraft  fairing,  shown  in  Figure  6, 
which  DLA  sold  for  $5,454  each. 


Figure  6.  B-52H  Aircraft  Fairing 
Source:  PDREP 


We  reviewed  the  PQDR  along  with  photos  of  the  deficient  part  that  the  Air  Force 
included  with  it.  We  identified  in  the  photos  that  the  contractor  identification  code 
inscribed  on  the  deficient  part  did  not  match  the  contractor  identification  code 
the  originator  cited  in  the  PQDR.  When  processing  the  report,  quality  assurance 
personnel  did  not  adequately  review  the  photos  and  the  details  of  the  complaint 
and  did  not  identify  the  discrepancy  with  the  contractor  identification  code.  As 
a  result,  the  quality  assurance  specialist  sent  the  defective  part  to  the  wrong 
contractor  who  rightly  claimed  that  it  did  not  manufacture  the  part.  The  quality 
assurance  specialist  considered  the  deficient  part  to  be  an  isolated  incident,  coded 
the  cause  of  the  deficiency  as  undetermined,  and  closed  the  PQDR  investigation. 

In  addition,  the  DLA  Aviation  supervisor  who  reviewed  and  approved  this  PQDR 
investigation  did  not  identify  the  discrepancy  before  authorizing  its  closure. 
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As  another  example,  we  reviewed  a  DLA  Aviation  quality  assurance  specialist’s 
investigation  of  a  Navy  PQDR  for  a  defective  AV-8  aircraft  duct  support,  as  shown 
in  Figure  7,  which  DLA  sold  for  $8,401  each. 


PDREP  identified  the  PQDR 
as  closed  and  included  as  an 
attachment  the  quality  assurance 
specialist’s  closing  letter 
concluding  that  the  incident  was 
an  isolated  case  and  that  the 
responsible  contractor  was  out 
of  business.  Although  the  quality 
assurance  specialist  ended  the 
PQDR  investigation  it  was  never 
formally  closed  in  EBS. 

We  found  that  the  quality 
assurance  specialist 
misinterpreted  a  number  inscribed  on  a  photo  of  the  part  as  a  contractor 
identification  number  for  another  contractor  that  was  out  of  business  and  ended 
the  investigation.  We  also  found  that  the  PQDR  originator  shipped  the  deficient 
part  to  the  correct  contractor  who  was  still  in  business.  As  a  result  of  our 
inquiries,  the  quality  assurance  specialist  reopened  the  investigation  in  PDREP. 
Overall,  the  PQDR  was  originated  in  February  2013  and  was  still  open  in  EBS  over 
17  months  after  its  original  submission.8 

Based  on  these  situations  and  the  facts  surrounding  the  other  examples  discussed 
throughout  this  report,  DLA  Aviation  could  improve  the  adequacy  of  its  quality 
assurance  supervisory  reviews. 

DLA  should  improve  the  adequacy  of  its  supervisory  reviews  and  include 
specific  procedures  necessary  when  it  reviews  PQDR  investigation  results 
and  associated  coding. 

Operational  Effectiveness  of  the  Product  Quality  Deficiency 
Report  Program  Not  Adequately  Monitored 

DLA  personnel  did  not  adequately  analyze  the  PQDR  data  to  identify  systemic 
problems,  trends,  and  causes  for  recurring  deficiencies  in  spare  part  quality  to 
improve  the  effectiveness  of  the  program.  DLA  headquarters  officials  stated  that 


8  DLA  policy  does  not  specify  a  time  limit  to  close  PQDR  investigations  but  our  analysis  of  the  sampled  PQDRs  found  that 
the  average  time  from  the  originator's  submission  until  the  closure  of  the  investigation  in  PDREP  was  272  days,  slightly 
over  9  months. 
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some  of  the  DLA  Supply  Chains  track  PQDR  processing.  However,  DLA  officials 
stated  that  efforts  were  primarily  focused  to  reduce  the  number  of  PQDRs  with 
causes  that  could  not  be  determined  and  increase  the  identification  of  root 
causes  for  quality  deficiencies.  DLA  headquarters  officials  stated  that  there  was 
no  guidance  or  established  metrics  to  measure  the  overall  effectiveness  of  the 
PQDR  program. 

Upon  resolving  the  coding  problems  identified  in  this  report,  DLA  should  develop 
procedures,  controls,  and  appropriate  metrics  to  identify  problems,  trends,  and 
recurring  deficiencies  in  spare  part  quality  to  improve  the  operational  effectiveness 
of  the  PQDR  program  and  provide  training  to  ensure  that  the  problems  this  report 
identified  do  not  reoccur. 


Process  Weaknesses  and  Software  Glitch  Caused 
Coding  Differences  Between  Key  Information  Systems 


The  cause  codes  assigned  in  EBS  and  PDREP  differed  for  17  of  the  52  PQDR 
investigations  we  reviewed  and  for  a  total  of  1,921  of  the 
9,347  PQDRs  that  DLA  Supply  Chains  closed  between 
August  2013  and  August  2014.  The  codes  differed  between 
the  two  systems  because  of  weaknesses  in  the  deficiency 
report  closing  process  and  because  the  PDREP  software 
was  erroneously  converting  certain  cause  codes. 


The  cause 
codes  assigned 
in  EBS  and  PDREP 
differed  for... 

11  of 
7  PQ] 


^BHF 


As  a  final  step  in  the  PQDR  process  after  the  action  point 
completes  its  investigation,  the  screening  point  closes 
the  PQDR.  To  accomplish  this,  screening  point  personnel  in 
organizations  who  use  JDRS  must  also  assign  codes  in  their  system  despite  a  DLA 
quality  assurance  specialist  having  already  done  so  in  EBS  and  the  codes  already 
being  transferred  to  PDREP.  However,  the  revised  codes  that  screening  point 
personnel  input  into  JDRS  do  not  transmit  back  to  EBS.  Instead,  PDREP  substitutes 
the  codes  screening  point  personnel  assign  in  JDRS  for  the  codes  that  the  DLA 
quality  assurance  specialists  previously  assigned  through  EBS. 


Screening  point  personnel  stated  that  they  did  not  see  the  DLA  action  point’s  codes 
but  instead  used  a  preliminary  closing  letter,  which  DLA  sent  to  them  as  a  courtesy, 
as  a  basis  to  select  the  codes  within  JDRS.  The  information  in  the  preliminary 
closing  letters  sometimes  changed  based  on  new  information  becoming  available 
and  also  often  did  not  clearly  indicate  DLA’s  coding  choices.  Therefore,  the 
screening  point  should  not  use  the  preliminary  closing  letter  to  assign  codes  and 
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dose  out  PQDRs  in  JDRS.  Instead,  the  screening  point  should  use  the  final  dosing 
letter  and  associated  codes  the  action  point  assigned  based  on  their  complete 
investigation  results.  When  this  does  not  occur,  coding  discrepancies  exist 
between  EBS  and  PDRER 

Figure  8  illustrates  the  process  and  how  PQDR  data  generally  flowed  between 
the  information  systems  used  to  process  PQDRs  where  DLA  Aviation  was  the 
action  point. 


Figure  8.  Process  Flow  of  PQDR  Coding  Data  Between  Systems 


Note:  The  process  starts  from  the  time  the  action  point  completes  its  investigation  until  the 
screening  point  formally  closes  the  PQDR  in  JDRS  and  updates  PDREP. 
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Our  review  of  a  DLA  Aviation  quality  assurance  specialist’s  investigation  of  an 
Air  Force  PQDR  for  deficient  E-3  aircraft  torque  tubes,  shown  in  Figure  9,  which 
DLA  sold  for  $923  each,  illustrates  the  deficiencies  with  this  process. 


The  PQDR  cited  an  urgent 
work  stoppage  and  requested 
DLA  screen  all  existing  stock 
for  additional  deficient  parts. 
Deficiencies  included  incorrect 
threads,  missing  cotter-pin 
holes,  primer  coating  instead 
of  cadmium  plating,  and  a 
lack  of  primer  inside  of  the 
tubes.  A  DCMA  investigation 
confirmed  that  the  contractor 
incorrectly  manufactured  the 
parts  and  the  quality  assurance 
specialist  appropriately  assigned 
the  cause  code  to  reflect  contractor  noncompliance  in  EBS.  However,  when 
closing  the  PQDR,  Air  Force  screening  point  personnel  subsequently  assigned 
Cause  Code  "U"  (undetermined)  in  JDRS  because  they  formed  their  conclusions 
from  a  preliminary  DLA  closing  letter  that  excluded  key  investigation  results. 
Consequently,  when  the  data  transferred  to  PDREP  it  replaced  the  correct 
DLA-assigned  code. 


Figure  9.  Torque  Tube  Assemblies  (Note:  the  top  part 
represents  the  defective  part) 

Source:  PDREP 


In  addition,  the  quality  assurance  specialist  appropriately  assigned  a  defect 
responsibility  code  of  "A"  (contractor)  in  EBS  to  reflect  contractor  responsibility 
for  the  defective  parts,  which  would  have  allowed  for  PPIRS  to  include  the  PQDR  in 
determining  the  contractor's  quality  rating.  However,  the  screening  point  changed 
the  defect  responsibility  code  to  “I"  (invalid)  and  caused  the  deficient  parts  to  not 
meet  the  PPIRS  criteria  for  calculating  the  contractor's  quality  rating.  Based  on 


our  inquiries,  the  quality  assurance  specialist  provided  the  Air  Force  screening 
point  with  an  updated  closing  letter  that  included  the  correct  information  and 
clearly  identified  contractor  noncompliance  as  the  root  cause 
for  this  deficiency.  Air  Force  screening  point  personnel 
subsequently  initiated  actions  to  correct  the  coding. 


Air  Force  officials  acknowledged  that  its  screening 
points  inappropriately  used  preliminary  closing 
letters  to  close  out  PQDRs.  They  explained  that  they 
sent  an  email  message  in  May  2014  to  their  screening 
points  to  remind  them  to  not  close  out  PQDRs  based  on 


Air  Force 
officials 

acknowledged  that 
its  screening  points 
inappropriately  used 
preliminary  closing 
letters  to  close 
out  PQDRs. 
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emails,  telephone  calls,  or  letters.  The  message  specified  that  action  points  should 
wait  for  the  incoming  application  process  interface  transaction  with  the  final 
closing  information  before  closing  the  PQDR  record  in  JDRS. 

In  addition,  DLA  and  PDREP  personnel  stated  that  they  planned  to  issue 
restrictions  on  how  systems  communicate  and  what  data  fields  could  be  changed. 
These  actions  should  limit  the  ability  of  JDRS  screening  points  to  change  DLA’s 
previously  established  findings  and  recommendations  of  investigation  codes. 

The  DLA  and  PDREP  officials  stated  that  estimated  changes  would  be  written 
in  the  summer  of  2015  and  implementation  is  estimated  to  occur  between 
December  2015  and  February  2016. 

We  also  identified  that  the  PDREP  software  was  erroneously  converting  the 
contract  error  Cause  Code  "C"  to  the  maintenance  error  Cause  Code  "M"  during 
the  transfer  process  from  EBS  to  PDREP.  This  impacted  859  of  the  9,347  PQDRs 
that  DLA  Supply  Chains  closed  between  August  2013  and  August  2014.  PDREP 
personnel  stated  that  the  glitch  was  caused  by  old  programming  code  which 
was  an  unintentional  remnant  of  the  interaction  between  formerly-used  systems 
where  a  Cause  Code  C  legitimately  needed  to  be  changed  to  a  Cause  Code  M.  They 
further  explained  that  the  code  conversion  process  was  not  properly  removed 
when  the  older  systems  were  phased  out,  which  led  to  the  existing  problems. 

When  we  brought  this  problem  to  the  attention  of  PDREP  system  personnel  they 
immediately  initiated  corrective  actions  to  fix  the  software  and  to  correct  PQDRs 
with  erroneous  codes. 

DLA  should  coordinate  with  the  PDREP  Program  Office  and  other  relevant 
organizations  to  ensure  that  the  planned  corrective  actions  are  implemented  and 
also  to  develop  periodic  review  procedures  to  ensure  that  the  codes  assigned  in 
EBS  are  consistent  with  the  codes  reflected  in  PDREP  for  the  same  PQDRs. 

Impact  of  Inadequate  Product  Quality  Deficiency 
Report  Processing 

Inadequate  PQDR  processing  leads  to  inaccurate  PQDR  data,  which  limits  the 
effectiveness  of  the  DoD  PQDR  program  and  prevents  meaningful  analysis  of 
the  primary  causes  of  spare  part  quality  deficiencies.  Specifically,  DLA  missed 
opportunities  to  identify  problems,  trends,  and  recurring  deficiencies  in  spare-part 
quality  and  to  improve  operational  effectiveness.  In  addition,  the  inaccurate  data 
weakened  DLA’s  ability  to  hold  poor  performing  contractors  who  provided  defective 
parts  accountable  because  contractor  evaluation  tools  such  as  PPIRS  contained 
incomplete  data.  Ultimately,  this  increases  the  risk  of  DoD  procuring  defective 
spare  parts  from  contractors,  which  impacts  warfighter  readiness  and  safety. 
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Missed  Opportunities  to  Improve  Operational  Effectiveness 

DLA  Regulation  4155.24  establishes  a  system  for  feedback  of  product  quality 
deficiency  data  and  provides  for  the  initial  reporting,  cause  correction,  and  status 
accounting  of  individual  product  quality  deficiencies.  The  Regulation  also  specifies 
that  DoD  components  should  use  the  data  gathered  from  the  PQDR  program  to 
identify  problems,  trends,  and  recurring  deficiencies  in  spare  part  quality.  In 
addition,  DoD  Instruction  4140. 019  requires  DoD  components  to  identify,  monitor, 
assess,  and  mitigate  (minimize  and  reduce)  potential  disruptions  within  the 
DoD  supply  chain,  and  requires  additional  life-cycle  management  controls  to  be 
developed,  applied,  and  maintained  to  guard  against  counterfeit  material  in  the 
DoD  supply  chain. 


DLA  had  a  wealth  of  product  quality  data  readily  available 
in  PDREP  that,  if  accurate,  could  be  used  to  fulfill  DoD 
requirements  and  improve  operational  effectiveness. 
Specifically,  DLA  had  data  available  that  identified 
problems  in  the  spare-part  manufacturing  process,  and 
identifies  trends  and  recurring  deficiencies  in  spare-part 
quality.  DLA  did  not  use  the  data  on  a  macro  level  to 
perform  any  trend  analysis  based  on  PQDR  coding.  For 
example,  PQDR  data,  if  accurate,  could  be  used  to  identify 
trends  during  overall  root-cause  analysis. 


Table  2  on  page  6  of  this  report  shows  our  audit  population  of  DLA  Aviation  PQDRs 
by  cause  code.  We  queried  this  data  directly  from  PDREP  and  sorted  it  to  identify 
the  PQDRs  by  cause  code.  The  data  show  that  14-percent  of  the  PQDRs  contain 
cause  codes  to  reflect  a  technical  data  package  error  or  a  contract  error.  This 
means  that  DLA  may  have  had  a  systemic  problem  with  its  contracts  and  associated 
technical  data  packages.  If  this  data  were  reliable,  DLA  could  further  investigate 
this  trend  and  potentially  identify  corrective  actions. 


Contractor  Evaluation  Tools  Contain  Incomplete  Data 

Inaccurate  PQDR  data  also  weakens  DLA’s  ability  to  hold  poor  performing 
contractors  responsible  for  providing  non-conforming  parts  because  contractor 
evaluation  tools  such  as  PPIRS  contain  inaccurate  or  incomplete  data.  PPIRS  is  a 
web-based,  enterprise  application  that  provides  timely  and  pertinent  contractor 
past  performance  information  to  the  DoD  and  Federal  acquisition  community  for 
use  in  making  source  selection  decisions.  PPIRS  assists  acquisition  officials  by 
serving  as  the  single  source  for  contractor  past  performance  data.  Contractors  are 


9  DoD  Instruction  4140.01,  "Supply  Chain  Management  Policy,"  December  14,  2011. 
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ranked  against  one  another  to  separate  the  contractors 
with  higher  quality  from  those  with  quality  problems.10 
If  PPIRS  contains  inaccurate  or  incomplete  quality 
data,  poor  performing  contractors  can  have  a  higher 
quality  ranking  than  they  deserve,  thus,  increasing 
the  likelihood  of  being  awarded  future  contracts  even 
though  they  previously  provided  defective  parts. 

Inadequate  Product  Quality  Deficiency 
Report  Processing  Could  Negatively  Impact 
the  Warfighter 


Ultimately,  inadequate  PQDR  processing  increases  the  risk  that  DoD  will  procure 
nonconforming  spare  parts  from  poor  performing  contractors  that  can  result  in 
work  stoppages  at  maintenance  facilities,  which  impacts  warfighter  readiness  and 
safety.  For  example,  the  complaint  related  to  the  torque  tube  PQDR  discussed 
earlier  in  this  report  stated  multiple  defects  including  plating  and  priming 
deficiencies,  incorrect  machining  of  thread  pitches,  and  missing  cotter-pin  holes. 
The  deficiencies  were  so  severe  for  this  item  that  they  caused  a  work  stoppage  on 
the  maintenance  line  for  the  door  the  torque  tube  assembly  was  used  to  repair. 


Overall,  PQDR  investigations  can  take  a  considerable  amount  of  time  to  complete, 
which  can  negatively  impact  warfighter  readiness.  We  analyzed  the  timeliness 
of  our  sampled  PQDRs  and  found  that  the  average  time  from  the  originator’s 
submission  until  the  investigation’s  closure  in  PDREP  was  272  days,  slightly 
over  9  months. 


In  addition,  defects  in  critical  safety  items  impact  warfighter  safety.  For  example, 
the  complaint  related  to  the  parachute  canopy  PQDR  discussed  earlier  in  this 
report  stated  that  two  parts  of  the  parachute  were  incorrectly  sown  together  which 
could  have  resulted  in  a  complete  blowout  of  the  canopy  apex  during  deployment. 
The  complaint  further  stated  that  it  could  increase  the  parachutists’  decent  rate, 
thus  resulting  in  severe  injury  or  loss  of  life. 


Conclusion 

DLA  missed  opportunities  to  increase  the  operational  effectiveness  of  the 
DoD  PQDR  program  and  decrease  the  risk  of  DoD  procuring  non-conforming  spare 
parts  from  poor  performing  contractors.  However,  if  DLA  addresses  the  problems 
we  identified  it  can  reduce  the  negative  impact  on  warfighter  readiness  and  safety. 
DLA  has  a  wealth  of  PQDR  data  readily  available  in  PDREP  that,  once  improved, 


10  See  Appendix  Cfor  details  on  PPIRS  quality  ratings. 
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could  assist  in  addressing  these  problems.  DLA  needs  to  analyze  the  spare-part 
quality  deficiencies  and  use  the  analysis  to  minimize  and  reduce  disruptions  within 
the  DoD  supply  chain.  In  addition,  DLA  will  be  in  a  better  position  to  hold  poor 
performing  contractors  responsible  through  restitution  or  declining  their  future 
award  potential  when  contractors  provide  nonconforming  parts  because  contractor 
evaluation  tools  will  contain  more  accurate  data.  Although  this  audit  focused 
primarily  on  DLA  Aviation,  our  recommendations  are  directed  at  DLA  Headquarters 
to  improve  DLA-wide  policy  and  controls.  DLA  should  ensure  that  the  revised 
policy  and  controls  are  implemented  at  all  DLA  Supply  Chains  that  process  PQDRS. 

Management  Comments  on  the  Finding 
and  Our  Response 

Although  not  required  to  comment,  the  Director,  Defense  Logistics  Agency  Logistics 
Operations  provided  the  following  comments  on  the  finding.  For  the  full  text  of  the 
Director’s  comments,  see  the  Management  Comments  section  of  the  report. 

Defense  Logistics  Agency  Comments  on  Adequacy  of  Quality 
Assurance  Guidance 

The  Director,  Defense  Logistics  Agency  Logistics  Operations,  agreed,  stating 
that  none  of  the  current  product  quality  deficiency  report  guidance  sufficiently 
expands  the  cause  code  definitions  and  provide  examples  of  when  to  use  them. 

The  Director  also  stated  that  they  provided  training  on  applying  the  proper  cause 
codes.  In  addition,  procurement  personnel  briefed  technical  quality  personnel  on 
the  Past  Performance  Information  Retrieval  System  and  the  importance  of  properly 
assigning  cause  codes  where  the  contractor  has  been  identified  through  the 
investigation  as  the  responsible  party. 

Defense  Logistics  Agency  Comments  on  Adequacy  of  Supervisory  Reviews 
The  Director,  Defense  Logistics  Agency  Logistics  Operations,  agreed,  stating  that 
in  many  cases  supervisors  focused  on  reviewing  the  content  of  the  product  quality 
deficiency  report  closing  letter  rather  than  the  specific  cause  code  assignments 
in  the  Enterprise  Business  System.  The  Director  also  stated  that  they  will  train 
supervisors  how  to  properly  review  product  quality  deficiency  reports  to  ensure 
the  assigned  codes  in  the  Enterprise  Business  System  correspond  to  the  findings 
and  conclusions  in  the  product  quality  deficiency  report  closing  letter. 
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Defense  Logistics  Agency  Comments  on  Adequacy  of  Program  Oversight 
The  Director,  Defense  Logistics  Agency  Logistics  Operations,  partially  agreed, 
stating  that  Defense  Logistics  Agency  Aviation  emphasizes  product  quality 
deficiency  report  reduction  and  provides  continuous  training  to  the  technical 
quality  community.  The  Director  agreed  that  they  can  certainly  improve  the 
overall  operational  effectiveness  of  the  program  by  identifying  problems, 
trends,  and  recurring  deficiencies.  The  Director  also  stated  that  they  have 
begun  to  sample  product  quality  deficiency  report  data  to  monitor  increases  in 
undetermined  root  causes  as  well  as  identify  instances  of  recurring  deficiencies. 

In  addition,  they  will  establish  a  suite  of  standardized  metrics  to  be  collected  and 
analyzed  throughout  the  Defense  Logistics  Agency  Enterprise  in  eWorkplace. 

Our  Response 

We  acknowledge  the  Director’s  comments  on  the  findings  and  appreciate  the 
actions  to  improve  Defense  Logistics  Agency  product  quality  deficiency  report 
processing.  Although  the  Director  partially  agreed  to  our  finding  on  the  adequacy 
of  program  oversight,  his  proposed  corrective  actions  sufficiently  address  the 
finding  and  associated  recommendation. 

Recommendations,  Management  Comments, 
and  Our  Response 

Recommendation  1 

We  recommend  that  the  Director,  Defense  Logistics  Agency  develop  a  plan 
of  action  with  milestones  to  improve  product  quality  deficiency  report 
processing  and  ensure  that  the  corrective  actions  are  implemented  at 
all  Defense  Logistics  Agency  Supply  Chains  that  process  product  quality 
deficiency  reports.  The  plan  should  address  the  problems  that  this  report 
identified  and: 

a.  Update  existing  guidance  for  quality  assurance  specialists  to: 

(1)  clarify  the  product  quality  deficiency  cause  code  definitions 
and  providing  specific  examples  of  situations  that  warrant 
the  use  of  each  specific  code; 

(2)  specify  how  the  coding  decisions  impact  contractors’  quality 
ratings  in  the  Past  Performance  Information  Retrieval 
System;  and 

(3)  address  the  sufficiency  of  supervisory  reviews  and  include 
specific  procedures  necessary  when  reviewing  the  results  of 
product  quality  deficiency  report  investigation. 
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Defense  Logistics  Agency  Comments 

The  Director,  Defense  Logistics  Agency  Logistics  Operations,  responding  for  the 
Director,  Defense  Logistics  Agency,  agreed,  stating  that  Defense  Logistics  Agency 
management  will  work  with  the  Supply  Chains  to  develop  and  implement  corrective 
actions  by  December  18,  2015.  Specifically,  these  actions  will  address  all  concerns 
in  the  Technical  Quality  Procedures  Deskbook,  the  Product  Quality  Job  Aid,  and  the 
Product  Quality  Supervisory  Review  Job  Aid. 

Our  Response 

Comments  from  Director  addressed  all  specifics  of  the  recommendation,  and  no 
further  comments  are  required. 

b.  Establish  the  integrity  of  the  product  quality  deficiency  reporting 
coding  then  develop  procedures,  controls  and  appropriate  metrics  to 
identify  problems,  trends,  and  recurring  deficiencies  in  spare  part 
quality  and  to  improve  the  operational  effectiveness  of  the  program 
and  provide  training  to  ensure  that  the  problems  this  report 
identified  do  not  reoccur. 

Defense  Logistics  Agency  Comments 

The  Director,  Defense  Logistics  Agency  Logistics  Operations,  responding  for 
the  Director,  Defense  Logistics  Agency,  agreed,  stating  that  they  will  pursue  a 
standardized  suite  of  product  quality  deficiency  report  metrics  for  the  Defense 
Logistics  Agency  eWorkplace  and  will  update  the  product  quality  deficiency  report 
course  to  ensure  that  the  concerns  in  this  report  are  addressed.  The  targeted 
completion  date  is  December  2015. 

Our  Response 

Comments  from  Director  addressed  all  specifics  of  the  recommendation,  and  no 
further  comments  are  required. 
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c.  Require  coordination  with  the  U.S.  Navy  Product  Data  Reporting  and 
Evaluation  Program  Office  and  other  relevant  organizations  to: 

(1)  ensure  planned  corrective  actions  are  implemented  for 
improved  communication  between  the  Enterprise  Business 
System,  Product  Data  Reporting  and  Evaluation  Program 
and  Joint  Deficiency  Reporting  System  and  will  sufficiently 
maintain  the  integrity  of  Defense  Logistics  Agency 
product  quality  deficiency  report  investigation  results  and 
associated  coding. 

(2)  develop  periodic  review  procedures  to  ensure  that  the 
integrity  of  Defense  Logistics  Agency  product  quality 
deficiency  report  investigations  and  associated  coding  in 
the  Enterprise  Business  System  is  maintained  in  the  Product 
Data  Reporting  and  Evaluation  Program. 

Defense  Logistics  Agency  Comments 

The  Director,  Defense  Logistics  Agency  Logistics  Operations,  responding  for  the 
Director,  Defense  Logistics  Agency,  agreed,  stating  that  they  will  continue  to  work 
with  the  DoD  Joint  Product  Quality  Deficiency  Report  Committee  and  functional 
experts  to  implement  and  update  supporting  systems  in  accordance  with  the 
Defense  Logistics  Manual  4000.25,  842P  Implementation  Plan.11  The  target  date 
for  system  updates  is  February  2017. 

Our  Response 

Comments  from  the  Director  addressed  all  specifics  of  Recommendation  l.c(l),  and 
no  further  comments  are  required.  Comments  from  the  Director  did  not  address 
all  specifics  of  Recommendation  l.c(2),  and  further  comments  are  required  to 
describe  how  the  periodic  review  procedures  will  be  developed. 


11  Defense  Logistics  Management  System  Supplement  842P,  PQDR  Data  Exchange,  replaces  the  system  unique  transactions 
currently  used  to  exchange  data.  This  enhancement  provides  the  DoD  Components  with  a  standard  electronic 
transmission  method  for  reporting  PQDRs  across  systems. 
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Appendix  A 

Scope  and  Methodology 

We  conducted  this  performance  audit  from  August  2014  through  April  2015 
in  accordance  with  generally  accepted  government  auditing  standards.  Those 
standards  require  that  we  plan  and  perform  the  audit  to  obtain  sufficient, 
appropriate  evidence  to  provide  a  reasonable  basis  for  our  findings  and  conclusions 
based  on  our  audit  objectives.  We  believe  that  the  evidence  obtained  provides  a 
reasonable  basis  for  our  findings  and  conclusions  based  on  our  audit  objectives. 

We  reviewed  the  following  applicable  guidance. 

•  DLA  Regulation  4155.24  /  Army  Regulation  702-7  /  Secretary  of  the 
Navy  Instruction  4855. 5A  /  Air  Force  Regulation  74-6,  "Product  Quality 
Deficiency  Report  Program,"  July  20,  1993 

•  DLA  Deskbook  Appendix  B35,  "Quality  Notifications,  Product  Quality 
Deficiency  Reports  (PQDR),"  October  30,  2013 

•  Enterprise  Business  Systems  Technical  and  Quality  Job  Aid,  Product 
Quality  Deficiency  Report  (PQDR)  Quality  Notifications  (QNs),  dated 
February  19,  2014 

We  contacted  personnel  from: 

•  DLA  Headquarters; 

•  DLA  Aviation; 

•  DLA  Land  and  Maritime; 

•  Naval  Sea  Systems  Command; 

•  Naval  Air  Systems  Command;  and 

•  Air  Force  Materiel  Command. 

We  conducted  a  site  visit  to  DLA  Aviation,  located  in  Richmond,  VA. 

We  obtained  a  population  of  1,102  PQDRs  which  were  closed  between  August  2013 
and  February  2014  where  DLA  Aviation  functioned  as  the  action  point.  From  this 
population,  we  selected  a  non-statistical  sample  of  68  PQDRs  to  evaluate. 

These  68  PQDRs  resulted  in  52  unique  investigations  because  originators 
sometimes  submit  multiple  PQDRs  for  the  same  deficiency  and  DLA  generally 
combines  investigations.  We  focused  primarily  on  those  identified  as  high  priority 
deficiencies  (category  I)  or  that  were  related  to  items  identified  as  critical  safety 
items.  We  also  considered  other  factors  such  as  price  of  the  parts  involved  and  the 
availability  of  exhibits. 
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We  reviewed  each  of  these  52  PQDR  investigations  to  determine  whether  the  DLA 
quality  assurance  specialist  had  performed  an  adequate  investigation  to  determine 
the  root  cause  of  the  reported  deficiency.  During  these  reviews,  we  interviewed 
the  quality  assurance  specialist  who  performed  the  investigation  as  well  as  their 
supervisor.  We  also  reviewed  documentation  of  the  investigation,  such  as  the  original 
complaint,  DLA  and  DCMA  investigation  findings,  pictures,  and  technical  drawings. 

We  obtained  a  larger  population  of  PQDR  coding  data  from  both  PDREP  and  EBS. 

This  population  includes  9,347  PQDRs  which  DLA  closed  between  August  2013  and 
August  2014  where  one  of  DLA’s  major  supply  centers  had  functioned  as  the  action 
point.  We  examined  this  entire  population  to  evaluate  whether  PQDR  coding  data 
were  consistent  between  PDREP  and  EBS. 

Use  of  Computer-Processed  Data 

We  used  computer-processed  data  from  PDREP  and  EBS.  We  obtained  data  from 
PDREP  in  the  form  of  PQDRs  closed  between  August  2013  and  February  2014. 

We  focused  on  PQDRs  where  DLA  Aviation  was  the  action  point  for  the  investigation. 
The  PQDRs  were  generally  initiated  in  either  PDREP  or  JDRS  by  Air  Force  or  Navy 
personnel,  and  then  transmitted  to  EBS.  To  test  the  reliability  of  the  PDREP  data, 
we  reviewed  the  PQDRs  and  validated  the  accuracy  of  the  investigation  results  and 
coding  listed  in  PDREP  by  interviewing  the  DLA  Aviation  quality  assurance  specialist 
that  performed  the  investigation  and  coded  the  PQDR  in  EBS. 

We  obtained  data  from  EBS  in  the  form  of  PQDR  investigation  coding  entered  into 
EBS  by  DLA  Aviation  quality  assurance  personnel.  To  test  the  reliability  of  the  EBS 
data,  we  interviewed  the  DLA  Aviation  quality  assurance  specialist  that  performed 
the  investigation  and  coded  the  PQDR  in  EBS,  and  we  also  compared  the  coding  for 
sampled  PQDRs  in  EBS  to  the  coding  in  PDREP. 

We  identified  unreliable  PQDR  coding  in  EBS  and  PDREP,  the  details  on  these 
deficiencies  are  provided  in  the  finding  section  of  this  report. 
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Prior  Coverage 

During  the  last  5  years,  the  Government  Accountability  Office  (GAO)  and  the 
Department  of  Defense  Inspector  General  (DoD  IG)  issued  two  reports  discussing 
Product  Data  Reporting  and  Evaluation  System  or  Product  Quality  Deficiency 
Reports.  Unrestricted  GAO  reports  can  be  accessed  at  http://www.gao.gov. 
Unrestricted  DoD  IG  reports  can  be  accessed  at  http://www.dodig.mil/pubs/index.cfm. 

GAO 

Report  No.  GAO-10-389,  "DoD  Should  Leverage  Ongoing  Initiatives  in  Developing  Its 
Program  to  Mitigate  Risk  of  Counterfeit  Parts,"  March  2010 

DoD  IG 

Report  No.  DODIG-2010-035,  "Defense  Logistics  Agency  Contracts  for  M2  Machine  Gun 
Spare  Parts  in  Support  of  Operations  in  Southwest  Asia,"  January  11,  2010 
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Appendix  B 

Product  Quality  Deficiency  Reporting  Process 
PQDR  Process  Walk  Through 

The  following  is  a  narrative  of  the  PQDR  process  when  DLA  is  the  action  point, 
which  is  illustrated  by  Figure  B  on  page  31. 

1.  A  user  at  a  military  department  repair  center  (originator)  discovers  a 
problem  with  a  DLA  managed  part  requisitioned  from  supply  and  initiates 
a  PQDR  in  the  appropriate  system. 

2.  Responsible  personnel  at  the  military  department  supply  activity, 
(screening  point)  are  notified  of  the  PQDR  and  verify  the  completeness 
and  validity  of  the  complaint,  and  assign  it  to  DLA  (action  point). 

3.  PDREP  forwards  the  PQDR  to  EBS. 

4.  A  quality  assurance  specialist  at  the  appropriate  DLA  field  activity 
responsible  for  the  part  will  receive  notification  in  their  workflow  of  a 
new  PQDR  and  begin  a  review  investigating  the  deficiency.11 

5.  The  quality  assurance  specialist  acknowledges  the  PQDR  and  then 
determines  whether  the  part  was  under  DCMA  purview  and  engages 
DCMA  as  a  support  point12  if  required  for  the  investigation. 

6.  The  contractor  is  notified  of  the  PQDR  and  requests  the  exhibit 
(defective  part)  from  the  support  point. 

7.  The  action  point  will  be  notified  from  the  support  point  that  an  exhibit 
is  required  by  the  contractor  and  they  will  then  request  the  exhibit  from 
the  originator. 

8.  The  originator  ships  the  exhibit  to  the  contractor. 

9.  The  contractor  under  supervision  of  the  support  point  will  open  the 
exhibit  and  perform  testing  of  the  exhibit  to  determine  whether  it 
is  defective. 

10.  After  testing,  the  support  point  and  contractor  will  determine  if  the 
deficiency  is  valid  and  send  their  findings  to  the  action  point.13 


11  If  the  PQDR  is  valid,  then  the  action  point  continues  with  investigation.  If  not,  they  will  close  the  PQDR  out  as 
being  invalid. 

12  DCMA  is  only  engaged  if  they  are  appropriate  support  point.  In  other  instances  the  action  point  may  go  straight  to  the 
contractor  or  they  may  obtain  assistance  from  DLA  Test  Labs. 

13  If  the  deficiency  was  valid,  then  the  action  point  will  determine  a  root  cause  and  assign  coding  in  EBS.  If  the  defect  was 
not  valid,  the  PQDR  will  be  closed  out. 
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11.  The  quality  assurance  specialist  (action  point)  will  review  the  findings 
from  the  test  and  perform  any  additional  analysis  needed  to  determine 
the  root  cause  of  the  deficiency.  He  then  assigns  codes  and  enters  text  to 
explain  the  cause  of  the  deficiency,  the  party  responsible  for  the  deficiency 
(contractor  or  Government),  actions  taken  to  correct  the  deficiency,  and 
the  disposition  of  the  defective  product. 

12.  The  action  point  will  prepare  a  closing  letter  based  on  the  findings  and 
analysis  and  issue  it  to  the  screening  point. 

13.  The  screening  point  will  review  the  closing  letter  and  if  they  agree  with 
the  findings,  forward  the  closing  letter  to  the  originator.14 

14.  The  originator  will  review  the  closing  letter,  if  they  agree  with  the 
findings  the  deficiency  is  considered  resolved.15 


14  If  the  screening  point  disagrees,  they  may  rebut  the  findings  of  the  investigation  and  PQDR  will  remain  open. 

15  The  originator  also  has  the  opportunity  to  review  the  findings  of  the  investigation  in  the  closing  letter  and  can  consider 
the  PQDR  resolved,  or  they  can  rebut  the  findings  and  not  close  out  the  PQDR. 
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Figure  B.  Typical  PQDR  Process  Flow  Chart 
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Appendix  C 

Past  Performance  Information  Retrieval  System 
Quality  Rating  Process 

PPIRS  Quality  Performance  Rating  Process 

PPIRS  provides  past  delivery  and  quality  performance  information  for  commodities 
including  contracts  under  the  thresholds  established  in  the  PPIRS  report  card 
system.  PPIRS  uses  past  quality  data  to  assign  a  quality  color  code  to  contractors 
for  each  Federal  Supply  Class  (FSC)  for  which  they  have  been  awarded  contracts. 

The  quality  formula  is: 

(Positive  weighted  data  minus  negative  weighted  data)  /  Contract  FSC  Line 
Item  Total 

If  there  were  no  delivery  data  available,  a  value  of  "1"  would  be  used  for  the  bottom 
quotient  (figure).  The  table  below  lists  the  quality  performance  records  PPIRS  used 
to  rate  contractor’s  quality  performance  and  the  weight  factors  for  each. 


Table  C.  Types  of  Contractor  Quality  Data  and  Associated  Weighting 


Record 

Service 

Positive 

Weight 

Negative  Weight 

Bulletins 

Navy 

N/A 

-1.0  (critical) 

-0.7  (major) 

Government  Industry  Data  Exchange  Program  Alerts 

All 

N/A 

-1.0  (critical) 

-0.7  (major) 

-0.2  (minor) 

Material  Inspection  Reports 

Navy 

+1 

-1.0  (critical) 

-0.7  (major) 

-0.2  (minor) 

PQDRs 

All 

N/A 

-1.0  (category  1) 
-0.7  (category  II) 

Surveys  (excluding  Preaward  Surveys) 

DCMA& 

Navy 

+0.7 

-0.7  (others) 

Test  Report  (1st  Article,  Production,  etc.) 

Navy 

+0.5 

-0.5 

PPIRS  assigns  a  color  to  each  FSC  for  which  there  is  quality  performance 
data.  Color  is  based  on  the  high  5  percent  in  the  commodity  (dark  blue),  next 
10  percent  (purple),  next  70  percent  (green),  next  10  percent  (yellow),  and  last 
5  percent  (red).  In  this  calculation,  the  companies  are  classified  based  on  quality 
performance  comparisons  for  all  competitors  within  an  FSC.  The  Figure  below 
illustrates  the  color  codes  that  PPIRS  assigns  to  contractors  based  on  quality. 
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Figure  C.  PPIRS  Quality  Performance  Rating  Color  Codes 


NOTE:  If  there  was  only  one  percentage  group  for  an  entire  FSC,  the  group  would  be  classified  as 
green.  If  a  contractor  had  delivery  data  but  no  quality  data  for  a  given  FSC,  that  contractor  would 
automatically  receive  a  green  rating  (Delivery  Green). 


For  example,  consider  three  contractors  producing  items  in  a  given  FSC.  Each 
contractor  had  a  total  of  three  contract  line  items.  Contractor  A  had  one  PQDR 
(category  I),  contractor  B  had  two  PQDRs  (category  I),  and  contractor  C  had 
three  PQDRs  (category  I).  Their  respective  quality  calculations  would  be  as  follows 
(Note:  the  examples  below  assume  no  positive  weighted  data  and  their  color  rating 
would  be  determined  based  on  the  distribution  of  all  contractors  within  the  FSC): 

•  Contractor  A:  1  PQDR:  (1  x  1.0  =  1.0) 

(0  -  1.0)  /  3  =  -0.333 

•  Contractor  B:  2  PQDRs:  (2  x  1.0  =  2.0) 

(0  -  2.0)  /  3  =  -0.667 

•  Contractor  C:  3  PQDRs:  (3  x  1.0  =-3.0) 

(0  -  3.0)  /  3  =  -1.000 
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Defense  Logistics  Agency  Comments 


DEFENSE  LOGISTICS  AGENCY 
HEADQUARTERS 
8725  JOHN  J.  KINGMAN  ROAD 
FORT  BELVGIR,  VIRGINIA  22060 622 1 


MAY272W5 


MEMORANDUM  FOR  DEPARTMENT  OF  DEFENSE  INSPECTOR  GENERAL 
{ACQUISITION,  PARTS,  and  INVENTORY) 


SUBJECT;  Response  to  DoD  IG  Draft  Report  "DLA  Can  Improve  Its  Product  Quality 
Deficiency  Report  Processing,”  (Project  No.  D201 4- DOOO AG-0205 .000) 


Attached  is  the  Defense  Logistics  Agency's  (DLA)  response  to  the  subject  Draft  Report, 
We  appreciate  the  opportunity  to  review  and  comment  on  the  findings  and  recommendations. 

TI  of  the  Inspector 


General, 


Rear  Admiral,  SC  USN 
Director  DLA  Logistics  Operations 


Attachment: 
As  stated 
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Defense  Logistics  Agency  Comments  (cont'd) 


Response  to  DoDIG  Report  on  “Defense  Logistics  Agency  Can  improve  its  Product 
Quality'  Deficiency  Report  Processing1',  (Project  No.  D2fll4-DOOOAG-O2Q5.00Q) 

As  requested,  \vc  me  providing  responses  to  the  general  comem  and  recommendations  contained  in 
the  subject  report, 

RECOMMENDATION  1 

We  recommend  that  the  Director,  Defense  Logistics  Agency  develop  a  plan  of  action 
with  milestones  to  improve  product  quality  deficiency  report  processing  and  ensure  that 
the  corrective  actions  are  implemented  at  all  Defense  Logistics  Agency  Supply  Chains 
that  process  product  quality  deficiency  reports.  The  plan  should  address  the  problems 
that  this  report  identified  and: 

Recommendation  ha.  Update  existing  guidance  for  quality  assurance  specialists  to: 

[1]  clarify'  the  product  quality  deficiency  cause  code  definitions  and  providing 
specific  examples  of  situations  that  warrant  the  use  of  each  specific  code; 

P)  specify  how  the  coding  decisions  impact  contractors’  quality  ratings  in  the  Past 
Performance  Information  Retrieval  System;  and 
(3)  address  the  sufficiency  of  supervisory  reviews  and  include  specific  procedures 
necessary  when  reviewing  the  results  of  a  product  quality  deficiency  report 
investigation. 

Response:  Concur.  DLA  J3  Management  will  work  with  the  DLA  Supply  Chains  to 
develop  and  implement  corrective  actions  specific  to  DLA  by  December  18,  2015; 

*  DLA  J3  management  will  address  all  concerns  and  provide  additional 

information  that  addresses  all  concerns  in  the  Technical  Quality  Procedures 
Desk  book,  the  Product  Quality  Job  Aid  and  the  Product  Quality  Supervisory 
Review  Job  Aid. 

Recommendation  LB.  Establish  the  integrity  of  the  product  quality  deficiency  reporting 
coding  then  develop  procedures,  controls,  and  appropriate  metrics  to  identify  problems, 
trends,  and  recurring  deficiencies  in  spare  part  quality  to  improve  the  operational 
effectiveness  of  the  program  and  provide  training  to  ensure  that  die  problems  this  report 
identified  do  not  reoccur. 

Response:  Concur.  J3  will  pursue  a  standardized  suite  of  product  quality  deficiency 
report  metrics  for  DLA  ©Workplace  and  will  update  the  Product  Quality  Deficiency 
Report  course  to  ensure  that  the  concerns  in  this  report  are  addressed.  Targeted 
completion  date:  December  2015. 

Recommendation  I.C.  Require  coordination  with  the  ILS,  Navy  Product  Data 
Reporting  and  Evaluation  Program  Office  and  other  relevant  organizations  to: 

(1)  ensure  planned  corrective  actions  are  implemented  lor  improved  communication 
between  the  Enterprise  Business  System,  Product  Data  Reporting  and  Evaluation 
Program  and  Joint  Deficiency  Reporting  System  and  will  sufficiently  maintain 

I 
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the  integrity  of  Defense  Logistics  Agency  product  quality  deficiency  report 
investigation  results  and  associated  coding,  and 
[2]  develop  periodic  review  procedures  to  ensure  that  the  integrity  of  Defense 
Logistics  Agency  product  quality  deficiency  report  investigations  and  associated 
coding  in  the  Enterprise  Business  System  is  maintained  in  the  Product  Data 
Reporting  and  Evaluation  Program. 

Response:  Concur.  DLA  will  continue  to  work  with  the  DoD  Joint  PQDR  Committee 
and  functional  experts  to  implement  and  update  supporting  systems  in  accordance  with 
the  DLM  4000,25,  S42P  Implementation  Plan,  Target  implementation  date  for  system 
updates;  February  2017 

******* 

The  following  summarises  report  findings  regarding  DLA  Aviation  and  Ihc 
respective  responses  from  DLA  Aviation: 

Finding 

DLA  Aviation  quality  assurance  personnel  conducted  adequate  investigations  for  product 
quality  deficiency  reports.  However  they  did  not  adequately  process  2 1  of  52  that  were  non- 
statistically  sampled  and  properly  code  them  to  reflect  the  mot  causes  of  the  deficiencies 
determined  by  their  investigations.  This  occurred  because: 

Finding  l 

*  Quality  assurance  personnel  lacked  sufficient  guidance  to  make  appropriate  coding  decisions 
and  did  not  have  a  complete  understanding  of  how  their  coding  actions  impacted  contractor's 
quality  ratings. 

Response:  Concur,  The  DLAR  4155.24  (authoritative  document  for  PQDR  reporting  and 
processing)  briefly  describes  the  Broad/ Detailed  Cause  Codes  and  the  Deficiency 
Responsibility  codes  for  PQDRs.  These  codes  arc  used  to  better  define  what  the  root  cause  of 
The  problem  was  and  to  determine  who  (contractor  or  government)  was  responsib!e/l table  for 
the  reported  deficiency.  However,  neither  DL  AR  41 55,24,  nor  any  of  the  current  PQDR 
guidance  sufficiently  expands  the  cause  code  definitions  and  provide  examples  of  when  to  use 
them.  DLA  Aviation  PQDR  BP  As  have  taken  the  action  to  provide  training  to  the  Product 
Specialists  on  applying  the  proper  cause  codes.  Additionally,  we  have  coordinated  with 
Procurement  -  to  brief  theTech/Quality  community  on  the  Past  Performance  Information 
Retrieval  System  i  PPIR5)  and  the  importance  of  properly  assigning  cause  codes  -  where  the 
contractor  has  heen  identified  through  the  investigation  as  the  rcspcinsible/fiablc  party. 

Finding  2 

*  Supervisors  failed  to  conduct  adequate  reviews  of  product  quality  deficiency  report 
investigations. 

Response:  Concur.  Supervisors,  in  many  eases  Focused  on  reviewing  the  content  of  the  PQDR 
closing  letter  rather  than  the  specific  cause  code  assignments  in  EBS,  If  the  closing  letter 
contained  all  of  the  appropriate  information  (represented  a  thorough  investigation  with  sound 
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recommendations),  the  supervisor  was  apt  to  approve  the  report  without  properly  reviewing  thti 
assigned  cause  codes.  We  have  since  provided  training  to  supervisors  on  how  to  properly 
review  PQDR  reports  to  ensure  the  assigned  codes  in  EBS  corresponds  to  the  findings  and 
conclusions  in  the  PQDR  dosing  letter.  The  training  is  ongoing  and  available  upon  request.  In 
addition,  supervisors  can  reference  the  PQDR  Supervisory  Review  Job  Aid. 

Finding  3 

■  Tile  product  quality  deficiency  report  program  lacked  adequate  oversight  to  improve 
opera  t  i  on  a  I  effect!  ven  ess. 

Response:  Partially  concur.  While  PQDR  reduction  is  emphasized;  DLA  Aviation  has  also 
continuously  provided  training  on  processing  PQDRs.  Walk-up/on  die  spot  training  as  well  as 
Division/ Group  training  is  available  to  the  Techmcal/Quality  community.  Yes,  we  can  certain!;, 
improve  the  overall  operational  effectiveness  of  the  program  by  identifying  problems,  trends, 
and  recurring  deficiencies,  To  this  end,  we  have  begun  to  sample  PQDR  data  to  monitor 
increases  in  undetermined  root  causes  as  well  as  identify  instances  of  recurring  deficiencies,  in 
addition,  J3  will  establish  a  suite  of  standardized  product  quality  deficiency  report  metrics  to 
be  collected  and  analyzed  throughout  the  DLA  Enterprise  in  eWorkplace* 
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Acronyms  and  Abbreviations 


DCMA  Defense  Contract  Management  Agency 
DLA  Defense  Logistics  Agency 
DoD  IG  Department  of  Defense  Inspector  General 
EBS  Enterprise  Business  System 
FSC  Federal  Supply  Class 
GAO  Government  Accountability  Office 
JDRS  Joint  Deficiency  Reporting  System 
PDREP  Product  Data  Reporting  and  Evaluation  Program 
PPIRS  Past  Performance  Information  Retrieval  System 
PQDR  Product  Quality  Deficiency  Report 
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Whistleblower  Protection 
U.S.  Department  of  Defense 


The  Whistleblower  Protection  Enhancement  Act  of  2012  requires 
the  Inspector  General  to  designate  a  Whistleblower  Protection 
Ombudsman  to  educate  agency  employees  about  prohibitions 
on  retaliation,  and  rights  and  remedies  against  retaliation  for 
protected  disclosures.  The  designated  ombudsman  is  the  DoD  Hotline 
Director.  For  more  information  on  your  rights  and  remedies  against 
retaliation,  visit  www.dodig.mil/programs/whistleblower. 


For  more  information  about  DoD  IG 
reports  or  activities,  please  contact  us: 

Congressional  Liaison 

congressional@dodig.mil;  703.604.8324 

Media  Contact 

public.affairs@dodig.mil;  703.604.8324 

Monthly  Update 

dodigconnect-request@listserve.com 

Reports  Mailing  List 

dodig_report@listserve.com 

Twitter 

twitter.com/DoD_IG 

DoD  Hotline 

dodig.mil/hotline 


